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^N ■ Surface operators in AA = 2 four-dimensional gauge theories are interesting half-BPS objects. 

CN \ These operators inherit the connection of gauge theory with the LiouviUe conformal field theory, 



which was discovered by Alday, Gaiotto and Tachikawa. Moreover it has been proposed that toric 
branes in the A-model topological strings lead to surface operators via the geometric engineering. 
We analyze the surface operators by making good use of topological string theory. Starting from 
^ ■ this point of view, we propose that the wave-function behavior of the topological open string am- 

C^ \ plitudes geometrically engineers the surface operator partition functions and the Gaiotto curves of 

corresponding gauge theories. We then study a peculiar feature that the surface operator corre- 
sponds to the insertion of the degenerate fields in the conformal field theory side. We show that 
this aspect can be realized as the geometric transition in topological string theory, and the insertion 
of a surface operator leads to the bubbling of the toric Calabi-Yau geometry. 



1 Introduction 

The Af = 2 four-dimensional gauge theories have provided many insights into the study of dynamics 
of gauge theory and string theory. In particular Seiberg and Witten [H [2] found that the low-energy 
dynamics of these theories is governed by the geometry of Riemann surfaces. Moreover Nekrasov 
[3] gave a microscopic derivation of the Seiberg- Witten geometry. 

Last year, a mysterious relation which relates M = 1 gauge theories in four dimensions to 
conformal field theories in two dimension was discovered in [3]. This relation originated in the 
M5-brane construction of A/" = 2 gauge theories which was developed in [5] . In this construction, 
a four-dimensional N = 2 superconformal gauge theory arises from compactification of M5-branes 
on a Riemann surface with punctures. Since a AA = 2 superconformal gauge theory of such a type 
is associated with a Riemann surface, it is very natural that there exists a relation between the 
Seiberg- Witten-Nekrasov solution of a AA = 2 superconformal gauge theory and the conformal field 
theory on the corresponding surface. Inspired by the development, Alday, Gaiotto and Tachikawa 
found the specific realization of the idea, which is called the AGT relation [4] : the Nekrasov partition 
function of a AA = 2 superconformal gauge theory is equal to the Liouville conformal block for the 
Riemann surface. 

The part of the extended objects, such as 'tHooft- Wilson line operators and surface operators, 
in the dual Liouville theory was explained in [U [71 E] : the M = 2 gauge theory in the presence of an 
elementary surface operator corresponds to the Liouville conformal block with an added insertion 
of a degenerate primary field <l>2,i(-2)- The expectation values of 'tHooft- Wilson line operators are 
then computed by using the monodromy operation for the conformal blocks. In this way the AGT 
relation can be extended to gauge theories in the presence of extended observables. 

In this paper we study the stringy realization of the AGT relation in the presence of elementary 
surface operators. By using M2-brane construction of surface operators [7], we can obtain surface 
operators from toric A-branes of topological string theory via the Ooguri-Vafa duality [9]. This 
is an extended version of the well-known geometric engineering of A/" = 2 gauge theories in four- 
dimensions [To]. The open topological string construction enables us to grasp some important 
properties of the AGT relation. 

In [1] it was found that the degenerate conformal block in the semi-classical limit gives the 
Seiberg- Witten curve in the Gaiotto form. We interpret this property string-theoretically by us- 
ing the wave-function behavior of the open topological strings [llj and mirror symmetry. Since 
this approach gives the differential equation for the expectation value of a surface operator, we 
can compute it very efficiently. Then we find that a Gaiotto curve relates to mirror Calabi-Yau 
corresponding to a AA = 2 gauge theory via four-dimensional limit or decoupling one of gravity. 



Recall the extended AGT relation [7] that the insertion of the degenerate field leads to the 
Nekrasov partition function in the presence of a surface operator, which we call the ramified in- 
stanton partition function. The degenerate confornial block is equal to a certain Nekrasov partition 
function for the specialized masses via the AGT relation. These facts imply that the Nekrasov par- 
tition function of a gauge theory gives the ramified instanton partition function of another gauge 
theory by degenerating physical mass parameters. We show that this is precisely the geomet- 
ric transition of the A-model topological strings under which branes change to the bubbling of a 
background Calabi-Yau geometry. 

The organization of this paper is as follows: In section 2 we study surface operators in the 
context of topological string theory. The role of the wave-function behavior in the AGT relation 
becomes clear. In section 3 we discuss the extended AGT conjecture for the insertion of surface 
operators from the viewpoint of topological string theory. We explain why the degenerate field 
insertion leads to the inclusion of a surface operator in the gauge theory side. The geometric 
transition gives a partial explanation to the question. In Appendix A we present a brief review on 
the Young diagrams and the Schur functions. We review the five-dimensional Nekrasov partition 
functions and the refined vertex in Appendix B. 

While preparing this paper, |12] which has some overlap with this paper appeared. 



2 Surface Operators and Topological Strings 

In this section we study the simple surface operators of A/" = 2 four dimensional gauge theories via 
the geometric engineering. We illustrate the relation between toric branes and surface operators 
with a simple example, which is called the Ts^o theory. This theory was studied extensively in 
[T3| El [E] . We show that the Gaiotto curve for the gauge theory comes of the wave function 
behavior of the corresponding open topological string amplitude. 

2.1 Toric brane and surface operator for Tio theory 

73,0(^1) theory, which is just the theory of four free hypermultiplets, are associated with the 
Riemann sphere with three punctures via the AGT relation. In this section, we study how surface 
operators arise from string theory setup. The key to stringy realization is the geometric engineering 
of A/" = 2 gauge theories. In [10] it was shown that Seiberg-Witten solutions are engineered from 
the genus-zero topological string theory on the corresponding Calabi-Yau geometries. The higher 
genus extension of the engineering was studied extensively in [161 [13 IISl Ell 1201 [21], and they 
found that all-genus topological string partition function leads to the Nekrasov partition functions 



through the topological vertex formalism [25]. The toric geometry which engineers the SU{2) gauge 
theory with four flavors is shown in the left side of Fig{T] 






Figure 1: The toric geometry which engineers the SU{2) gauge theory with four flavors (left) and 
the corresponding Gaiotto curve (right). This geometry is constructed by gluing two geometries 
that engineers the four free hypermultiplets theory respectively. 

Let us compactify M5-branes on a Riemann surface in order to construct N = 2 gauge theory 
125^15]. The SU{2) gauge theory with four flavors 71,0 (^i) comes of the Riemann surface with four 
punctures. We turn off the gauge coupling to get free theory: this is done by tearing the surface into 
spheres with three punctures. Since the Riemann sphere with four punctures degenerates into two 
Riemann surfaces with three punctures, we obtain the toric geometry associated with this 73,o(^i) 
theory by decomposing the toric geometry of 71,0 (^i) as Fig{T] We thus employ the half-geometry 
of Figd] as the toric geometry for the 73,0 theory. 

Let us consider the open A-model topological strings on this geometry in the presence of a 
toric A-brane on a non-compact Lagrangian submanifold. Let U be the holonomy matrix for the 
non-dynamical gauge field on the A-brane. The open A-model amplitude for Figgis then given by 



Z°P-^ = Y^TtyU ■ ZY{Qi,Q2,Q3;q), 



(2.1) 



and it is easy to compute Zy{Qi, Q2, Qf ; q) by using the topological vertex [22]. We will compute 
the refined version of the amplitude in the next section. Since we insert a single A-brane, the 
holonomy is one by one matrix U = diag(z). The trace for the representation R is then non- 
zero only for the Young diagrams with one column Y = [1"], and we obtain the following simple 
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Figure 2: The open A-model topological strings with a toric A-brane on the leg. The background 
geometry is the toric Calabi-Yau three-fold which engineers the 75,0 theory. The dotted line here 
denotes the non-compact Lagrangian branes. The Young diagram Y labels the representations of 
the Wilson loop for the boundary of the world-sheet. 



expression: 



z-p--(z) = J2^'' ■ z^''\Qi^Q2,Q3;> 



n=0 



ii (1_5.)(1_^*-1Q,Q3) 



j=l 



We study the gauge theory interpretation of this open string amplitude. 



(2.2) 
(2.3) 



2.2 Gaiotto curves via wave function behavior 

Let us introduce the following parametrization: 



z = e ", g = e ^% 



(2.4) 



where g^ is the topological string coupling constant. Since g " is the shift operator g "e " = qe ", 
it acts on the partition function as follows: 



(n) 



1 - q^-'Qi){l - q^~'QiQ2Q3) „_„„_(„_i),^(„_i)_ 



-e e 



Therefore, the operator 1 — g" " shifts the length of a column [1*^]. This relation gives the following 
differential equation for the open string partition function 

((1 - QiQzq-^--^){l - g-^") - z{l - Qiq-^-){l - QiQ2Qzq~^-)) Z°^^'\z) = 0. (2.5) 



This characteristic of the open topological string amplitudes is called the wave function behavior 
[11^ [23l I24j . This feature relates to the mirror B-model of the system: the mirror of A-brane is 
the fermion ip{z) in the two-dimensional Kodaira-Spencer theory. Then quantization of such a 
mirror system on a Riemann surface leads to the wave equation H ■ {tp) = 0. The Hamiltonian, 
or the Schroringer difference operator H, is associated with the local mirror Calabi-Yau geometry 
xy + H{u, v) = 0. By applying the wave equation for the following genus expansion of the wave 
function 

■ 1 



Z°P°"(z) = exp 



9s 



(2.6) 



where each term W(g\ is determined recursively. We study the recursion relation from the viewpoint 
of the geometric engineering of the gauge theories. 

Field theory limit 

Let h be the self-dual fi-background in four dimensions ei = — e2 = h. Then the gauge theory 
parameters are introduced by 

Q = e-«^ ^, = 6-^"^% (2.7) 

where R and rua are the fifth-dimensional radius and masses respectively. We take the four- 
dimensional limit of the open string amplitude as follows: 

W(„)(z)=lim i?--iVF(„)(z), (2.8) 

where VF(o) = W. Then the four-dimensional partition function, which corresponds to the Nekrasov 
partition function in the presence of a surface operator, takes the following form: 



^(z) = lim Z°P^"(z) = exp 



-W(z)+W(i)(z) + nW(2) + 



_ y^ n TT (j^ - h + mi){jh - h + mi+m2 + ma) 
~ ^^ J=i ■?'^(-?'^ -h + mi+ms) 

Notice that this is a fundamental solution of the hypergeometric differential equation near z = 

(mi mi+m2 + m3 mi + m^ \ 

^^^ VlT' h ' ^^' V ■ ^'•'°) 

This coincides with the sphere conformal block with the insertion of two primary fields and a 
degenerate field. 

We want to extract the information of the Seiberg-Witten curve, or Gaiotto curve, from the 
partition function. The wave function behavior then plays a key role. Let us expand the differential 



equation in R 

((mi +m3-hdu- h^ )(-/i5„ H ) 

- z{mi -hdu + --- ){mi +m2+m3-hdu + --- ))Z°P''°(z) = 0, (2.11) 

where the dots • • • represent terms of order 0{R). In the field theory Umit, the wave equation (|2.5p 
takes the following form 

[(1 - z)h^du^- {{nil + "1-3 - h) - z{2mi + ?ri2 + m^)) hdu 

- zmi{mi + 1712+1713)]^ {z) =0. (2.12) 

This four-dimensional open string wave function is exactly the hypergeometric differential equation 
whose solution is (12. 9p . The lesson here is that the wave function behavior of the open topological 
strings gives the null state condition on the degenerate conformal blocks which are the ramified 
instanton partition functions via the AGT relation. This construction, or the geometric engineering, 
of the the ramified Nekrasov partition functions implies that the Gaiotto curve can be derived from 
the mirror Riemann surface via geometric engineering. 

The wave function (I2.12p in the leading order h^ constraints the superpotential W, which is 
four-dimensional limit of the disk amplitude, as follows 

(1 - z)duyV{zf + ((mi + ms) - z{2mi + m2 + mg)) a„>V(z) - zmi{mi + m2 + mg) = 0. (2.13) 

It is easy to solve this "loop equation" as follows 



^ \ al{z'^ - z') + al{l - z') + a 



z' 



W(z) = a2log2 + a3log(l-z)± / ^ dz', (2.14) 

] z'{\ - z') 

where we introduce the CFT parameters 

mi = — Qi + a2 — "3) m2 = ai — q;2 — 03, ms = ai + 02 + 03- (2-15) 

The logarithmic terms of the solution are just the overall factor z"(l — z)" . The Gaiotto curve is 
then given by 9uW(o) = 4>2{z) as follows: 



Uz) = ^^i^^'-^) + ^}^\-^) + '-i\ (2.16) 

z{l Z) 

This is the well-known Gaiotto curve for the sphere with three punctures, or the Af = 2 theory of 
four free hypermultiplets 73,0 [131 [T4] . 



Generic ^-background and Nekrasov-Shatashivili limit 

We can also compute the wave function for the generic 0-background ei 7^ — £2- We call the 
corresponding extension of topological string theory "the refinement" [26]. The refined topological 
vertex computation in the next section leads to the refined partition function 

The difference equation follows from this expression of the refined partition function: 

:i - QiQ3'7~^"i"')(l - g~^") - z{l - Qi<?-^")(1 - QiQ2Q3q-^'')) ^°P™(^) = 0. (2.18) 



This difference equation is the five-dimensional lift of the differential equation derived from the null 
condition of the degenerate field |141ll5j. It would be interesting to study such difference equations 
by employing the five-dimensional version of the AGT relation and the (7-Virasoro conformal blocks 

An important feature of the refined wave-equation is that it depends almost exclusively on q (or 
t) for the brane on an un-preferred toric leg^ |261I28]. Recall that ei^2 are the rotational parameters 
of two distinct planes in C^, and a sueface operator fills one plane. It is therefore natural that the 
corresponding toric brane senses one-half of these parameters. In the following, we study the wave- 
function in the Nekrasov-Shatashivili limit t — )• 1 [29]. Now let us assume that the wave function in 
the limit is almost equal to the unrefined one. Then we can use the following mirror wave-equation 
for the toric brane [TT] in place of the differential equation in the Nekrasov-Shatashivili limit 

1/(6^,6^) •Z°P™(z)=0, (2.19) 

where [x,p] = gg. Note that the polynomial H{eP,e^) = defines the mirror curve of the toric 
geometry. We expect that this equation essentially leads to the Schrodinger equation for the 
corresponding integrable system. 

Let us take the SU{2) pure super Yang-Mills theory for example. The quadrangle toric web- 
diagram describes the Calabi-Yau three- fold which engineers the gauge theory (see FigU|). Since we 
obtain the mirror geometry by fatting the toric diagram off, the genus of the corresponding curve 
is one. The local mirror symmetry [TO] implies the defining equation 

-z + - + y^ + ciy + Co = 0, (2.20) 



^In this paper we study toric branes on an un-preferred leg of the refined topological vertex. Branes placed on 
the preferred leg lead to the homological link invariants through the geometric transition [28) . 



where we fix the irrelevant coefficients by re-normalization. The two Kahler moduli coordinates of 
B-model are given by the mirror map 

ZB = ^, = e-*« + • • • , ZF = ^, = e"*^ + • • • . (2.21) 

The quantum corrections of zb, which is denoted by the dots, are irrelevant in the field theory limit 
of interest, since it is the large radius limit cq = 1/ I^B? and i? — )• 0. We also fix 

1 Aci2 
in order to recover the Seiberg-Witten curve: 




(2.23) 



Then the field theory limit i? — )■ implies 



A^ (z + - j +/ + n = 0. (2.24) 

where we take y = —e~^^\/l/B? — u. In this limit, the quantization condition of the mirror curve 
becomes [x,p\ = ei where z = e^ . This is precisely the Schrodinger equation for the sine-Gordon 
model 

ei2^ + A2(e" + e-")+n')^ = 0. (2.25) 

In |30 1 I3H [32 } I33|. the authors have proposed that the wave-functionof the sine-Gordon system is 
equal to the ramified Nekrasov partition function in the Nekrasov-Shatashivili limit. Our argument 
gives a naive derivation of the conjecture from the perspective of the geometric engineering. 

Notice that we have assumed that the open string amplitude in the limit t = 1 satisfies essentially 
the same equation as the self-dual case q = t. It is not evident that such an observation is true 
for more complicated cases. In fact the dual-CFT computation in [33] shows that the differential 
equations for SU{2) gauge theories also depend on €2- This fact suggests that we have to modify 
our naive derivation. Thus it is very important to improve our sketch of stringy derivation by 
formulating the refined version of the B-model and the corresponding integrable systems. 

2.3 Geometric engineering of surface operators 

So far, we have assumed that an open topological A-model amplitude gives the corresponding 
Nekrasov partition function in the presence of a surface operator. We then verify this expectation 
by combining the stringy construction of surface operators with string dualities. 



Let us start with the M-theory construction of surface operators. Recall that a gauge theory of 
[5] can be realized as the world-volume theory of M5-branes conipactified on a punctured Riemann 
surface. A surface operators is then engineered by a semi-infinite transverse M2-brane. Let x, y G C 
be two complex combination of four coordinates x^,x^,x^ and x^. Then the holomolorphic curve 
which M5-branes wrap is described by S : H{x, y) = 0. We summarize the configuration as follows: 



M5 
M2 
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(2.26) 



Let us move on to Type IIA superstring setup. The dimensional reduction along the M-circle x^^ 
leads to the type IIA string theory with NS5-branes on S and a transverse D2-brane. 

Recall that the transversal T-duality replaces A^ NS5-branes to a local A^^i singularity [Ml [35]. 
After T-dualizing, the circle fiber degenerates on the locus on which the NS5-branes were placed. 
Thus, taking the T-duality along the circle transverse to S [36], we obtain the type IIB string 
theory on the following Calabi-Yau geometry 



uv + H{x,y) = 0. 



(2.27) 



Then the D2 brane changes into a semi-infinite D3-brane by the T-duality. This D3-brane wraps 
the following two-cycle parametrized a complex parameter H{xo,yo) = 



u = 0, t; = arbitrary, i?(xo,yo) = 0. 



(2.28) 



The cylinder fiber uv = —H over the xy-plane degenerates on the curve H{x,y) = into two 
cones corresponding to uv = 0, and the D3-brane wraps one of them as illustrated in Figj3j The 




Figure 3: The D3 brane wraps the non-compact two-cycle whose circle fibration degenerates on the 
base Riemann surface H{x,y) = 0. 



parameter (xq, i/q) is a modulus of the D-brane, which corresponds to a point on the base Riemann 
surface S. This modulus parametrizes an elementary surface operator. 

Finally we apply the mirror symmetry on this type IIB setup. The mirror of the geometry (j2.27p 
is well-studied [37]: it is a toric Calabi-Yau three- fold whose toric diagram is the skeleton which 
is obtained by thinning down the Riemann surface S. The mirror brane of (j2.28p is also studied 
in |381I39]: it is the D4-brane which wraps the non-compact Lagrangian three-cycle corresponding 
to the semi-infinite line on the toric diagram as FigJH This D4-brane also fills the subspace of the 
space-time M? C M^. This is precisely the setup of the Ooguri-Vafa duality [9]. As was explained in 



\yiy 



Figure 4: The toric diagram of the Calabi-Yau mirror to uv + H{x, y) = 0. The toric skeleton is 
just the thinned Riemann surface H{x, y) = 0. 

[H], we can compute the effective F-term in the presence of the D4-brane by counting the open world- 
sheet instantons via the A-model open topological strings. The open topological string amplitude 
for a toric Calabi-Yau such as Figgis computed by using the topological vertex formalism. Note 
that the topological vertex for the closed topological strings on the engineering toric Calabi-Yau 
gives the corresponding Nekrasov partition function. Therefore, by introducing the Lagrangian 
brane, we can extend this idea to the instanton counting in the presence of surface operators. In 
this way string dualities verify the observation that the open topological string amplitude for the 
toric Calabi-Yau which engineers a gauge theory leads to the ramified instanton partition function. 
From this stringy realization of surface operator, we can also explain why the insertion of the 
degenerate field $2,1 leads to a ramified instanton partition function. Recall the discussion of |43| : 
the collective fields for the matrix corresponding to the B-model imply the Liouville field. The 
mirror of N toric A-brane is A^ B-brane whose matrix description is the insertion of 



VNiz) = det($ - z 



,13 N 



-Nei<t>(z)/2 



(2.29) 



where $ is the matrix field, (f) is the Liouville field and /3 = \J —e^j^x 03 • Here we use the CFT 
description of matrix model -y/— eie2<90 = tr (z — $)~^. A single brane A^ = 1 thus leads to the 
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degenerate field <I>2,i = ^~^'^ of interest. In this way we can show that the degenerate field leads to 
the surface operator via the string dualities. See [Ml HSJ HSl HT] for details of the /3-ensemble which 
is the B-model in the generic ri-background. 

In the next section, we study the AGT relation by using this topological string description of 
the surface operators. We employ the refined topological vertex in order to deal with the generic 
ri-background ei^2- 

3 Geometric Transition and AGT relation 

In this section we explain why the degenerate field insertion gives the instanton partition function 
in the presence of a surface operator. We employ the concept of the geometric transition |40 | I4 H H2] 
for the purpose of studying this aspect of the AGT relation. 

3.1 Geometric engineering of surface operators 

Let us recall the two dual constructions of surface operators: the insertion of degenerate fields in the 
CFT side and the open string version of the geometric engineering with toric branes of topological 
strings. By comparing these two approachs, we reveal an interesting aspect of the AGT relation 
which is a gauge theory version of the open/closed duality. 

Surface operators from degenerate AGT relations 

By using the AGT relation, we can compute an instanton partition function from the corresponding 
conformal block of the Liouville CFT. The Nekrasov partition function in the presence of a surface 
operator also can be calculated by introducing an additional degenerate field in the corresponding 
conformal block. This result implies the following characteristic: by the specialization of parameters 




72,1 7i,i 

Figure 5: 72, i v.s. 7i,i : torus with two punctures and torus with a puncture and a degenerate 
field insertion. The X denotes the insertion of a degenerate field. 

which corresponds to change a primary to a degenerate field in the Liouville CFT side, an ordinary 
Nekrasov partition function leads to a surface operator partition function of another gauge theory. 

11 



See FiglS for a typical example. The torus with two primary fields (pmictures) corresponding to 
the so-called Ai quiver gauge theory (a.k.a. the 72, i theory). On the one hand, the torus with a 
primary and a degenerate fields, which is the right hand side of FiglSj describes the M = 2* theory 
with an elementary surface operator. We can therefore obtain the surface operator of A/" = 2* 
theory by adjusting the parameters of the Nekrasov partition function of the Ai quiver to the 
insertion of the degenerate field. 

It is very unclear why surface operators emerge only by adjusting physical parameters of a 
gauge theory. In this section we explain this phenomena from the perspective of the open/closed 
duality. Before studying it, we mention to the stringy version of this characteristic. 



Surface operators from open topological strings 

As we saw in the previous section, surface operators emerge from the topological A-branes via the 
geometric engineering. Meanwhile we can compute a partition function for surface operator from a 
Nekrasov partition function by choosing degenerate parameters of a gauge theory. The open string 
amplitude which engineers a surface operator therefore comes of an closed string partition function 
by adjusting parameters. This stringy situation corresponding to the A/" = 2* theory (Figj5]) is 




rv 




7-2,1 



^1, 



Figure 6: The toric geometry which engineers the SU{2) x SU(2) Ai quiver gauge theory Ti.i and 
brane insertion in the 7i,i geometry. 

illustrated in FigEl Then a question arises: why do the open strings be recovered from the closed 
strings by choosing degenerate parameters? We will give an answer to this question by utilizing 
the open/closed duality, or the geometric transition, of the topological string theory. 
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3.2 Tl.o '• SU{2) gauge theory with 4 flavors 

Let us consider the simple example, SU{2) gauge theory with 4 flavors (a.k.a the 71,0 [^i] theory). 
This theory is related to the sphere with four functures through the M5-brane construction. In 
order to introduce a surface operator, we need to add a degenerate field in the CFT side. We then 
obtain the sphere with five functures, which corresponds to the 75,o[^i] theory. We therefore study 
the geometric engineering of the Ts.oi^i] theory. 



75,0 : SU(2) X SU(2) quiver gauge theory 

The 75,0 [^i] theory is the SU{2) x SU(2) quiver gauge theory with a bifundamental hypermultiplets 
and two fundamentals for each SU{2). This theory is engineered by the toric Calabi-Yau geometry 
which is illustrated in FigJTl 

The topological string partition function for the generic fi-background is computed by using the 
refined topological vertex. We compute it by dividing this geometry into three parts in order to get 
the expression of the Nekrasov-type. See Appendix B for the definition of the refined topological 
vertex. We choose the horizontal direction for the preferred one of the refined vertex as in [261 [48] . 



Y 



>K 



V 




Figure 7: The toric geometry which engineers the SU{2) x SU{2) linear quiver gauge theory 75, o- 

Let us compute the partial amplitude in the left side of FigJT) We attach Young diagrams Yi^2 
on the two chopped edges. Since all the bundles over the local curve CP inside the geometry are 
0{—\) © 0{—\), the framing numbers are all zero and we need not to insert any framing factors. 
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The topological string amplitude is then given by 

2 2 

= \{q^^^ZYSt.q) E Sj,^T{Q^,Qf,t-Pq~^-) 
a=l Ri,V,W 

X Sn,/vi-QJ'q~'')SR,/vi-^/q^tQht-n 

X SR,{-Qm,q~')- (3.1) 

This is a typical strip geometry partition function which is ubiquitous in the geometric engineering 
of the superconfomal gauge theories. We utilize the refined vertex on a strip [491 I48j for evaluating 
such a summation of the Schur functions. We thus obtain the partial amplitude for the left side of 

Figm 

2 9 

Z^YuY2(t,q; Qrm,Qm2,Qh) = Y[q^~ZYAt,q) 

a=l 
*J=1 



(i-QmiQ/ii-^^+^g-^^^+^-i; 






oo 
X 



where the Kaher parameters are identified with the gauge theory parameters through 

Qi2:=Q™iQ/i=e-2^», (3.3) 

Here a is the Coulomb moduli parameter of the gauge factor corresponding to the left side of the 
toric blocks and mi^2 are the masses of fundamentals for the SU{2) factor. 

The partial amplitude for the middle of FigjT] is also a typical strip amplitude. We can write 
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this amplitude with the refined vertex as 

Ri 



\R3\ 
m2) 



X 



C#Ri Y^T{q, i)CR2 iJi^Vl/2'^(*' l)^R2^R-i ^2^(9' 0C'^R3^H/i^(9' 



1 1 -^g II 



a=l Ri,V,W 



X 5^2/iy (*"''' q~') Sr,/w [Vqftt-'q-^^ 

xS^.^Ti-Q^.^/tfqt-'^^q-p). (3.6) 



Taking the summation over the Young diagrams, we obtain the following expression for the ampli- 
tude: 



a=l 

oo 



X 



(3.7) 
We rewrite the Kaher parameters as 

Qi2:=Qm,i(0/i=e-''^", (3.8) 

Q„^ = e-«(«-«-™)g-i/2ii/2^ (39) 

Q^^ = e-R(-a+S-m)^-l/2^1/2^ (3_-^q) 

where m is the mass of the bifundamental matter and a is the Coulomb moduli parameter of the 
gauge factor corresponding to the right side of the toric diagram. This factor Z^^ therefore gives 
the instanton measure for the bifundamental via the geometric enginnering. 

The right side of the partition function is essentially the same as Z^ , since the refined vertex 
implies 



r^ 



X CR^<pwi{q,t)Cj^^Tj^^^{t,q)Cj^,^j^^T^^{q,t)Ci^^T^^{t,q), (3.11) 
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where 

Ql2:=QmlQ^=e-2^^ (3.12) 

Then we can rewrite it into Z^ as follows: 

Z^Wx^v-, {t,q; a, mi, 7712) = Z^Wi,W2 {Q,t] a, -mi, -7712) ■ (3.15) 

The full partition function is obtained by gluing these three partial amplitudes. By using 
formulae of Appendix B, we can recast it into the instanton measures. Then the normalized 
partition function is precisely the Nekrasov partition function for the 75,o[^i] theory: 



y- gin|+|y2lgimi+iW2l ^^^^'^^^^''^' "' "^i'"^2)^^V,,y,,tyT^^T(t,g)Z^tyT^tyT(g,t; a,mi,m2) 

^ Z^4,4,{t,q; a, mi,m2) Z'^4, 4, 4,^{t,q)Z^^ 4,{q,t; a, -mi, -7712) 

1^1,2, M'1,2 

= Y. MQl^^l+l^2lQl^^l+l^^lz.ect.,5D(a,y;t,(?) J] ^fnnd.M^,Y,mf;t,q) 

11,2,1^1,2 /=1,2 

X 2bifund.,5L>(a, Y; a, W; m; t, g)zvect.,5D(«, W'; i, q) JJ ^;fund.,5D(«, W^, "^/; i, 9), (3.16) 

/=1,2 

where M is a monomial of q, t and Q*, which is irrelevant when taking the four dimensional limit. 
In fact, by taking the 4-dimensional field theory limit, this topological string partition function 
recovers the Nekrasov instanton partition function. 

Degenerate Kaher moduli and bubbling 

Now let us move on to the emergence of a surface operator. The AGT relation suggests that 
the following specialization which corresponds to the degeneration of a primary field leads to the 
Nekrasov partition function in the presence of a surface operator: 

a = —rhi = 7712 + ei. (3-17) 
Then the Kaher parameters of the toric geometry take the quantized values 

Q„^ = g-3/2^1/2 ^ ^ ^-1 ^^^^ q = t), (3.18) 

g^^ = ^-1/2^1/2 ( ^ ;L ^j^gj^ q = t). (3.19) 

Note that tmi and tm2 here are essentially and Qs, where gs is the topological string coupling 
constant. The geometric transition will explain this quantization of the Kaher parameters. 
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This degeneration of the Kaher parameters siniphfies the topological string partition function. 
Let us focus on the instanton measure coming from the fundamentals: 



Y\_ Z{und.,5D{a,W,mf;t,q) 

/=1,2 



(i,j)eWi ii,j)eW2 



(3.20) 



From these factors, the Young diagram Wi must not contain the box (1,2) and W2 must be the 
empty Young diagram <P in order to give a non-zero contribution to the partition function. This 
means that the Young diagrams W have to take the following form: 



Wi = [l^] (n = 0,l,2,---), W2=<P. 



(3.21) 



These degenerate parameters thus cut off the toric geometry of the original theory as FigJSj This 
toric geometry is very resemble to that of the SU{2) gauge theory with four flavors. The reason of 



it will soon be clear. 



M^ 



fV^tl/2 



YY 



<P <P' 



g-1/2^1/2 



Figure 8: The toric geometry which engineers the SU{2) gauge theory with four flavors in the 
presence of the simple surface operator. 

The limitation W = ([1"], <?) simplifies the partition function for the 75,o[^i] theory. Due to the 
AGT relation, the resulting partition function is precisely the ramified instanton partition function 
for the Nf = 4 theory. In this way the degeneration of parameters gives the five- dimensional 
Nekrasov partition for the SU{2) gauge theory with four flavors in the presence of a surface operator: 

>'i,2,n /=1,2 



X ^bifund.,5D(a,>";(-?7ii, mi),([l"],^);m;i,g) 

n 

X ^vect.,5D((-mi, mi), ([r], ^); t, q) n(l - e2^'^if-i)(l - f-\_ 



(3.22) 



i=l 



We can recast it into the Nekrasov partition function for the Nf = 4 theory with an added insertion 
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of the factor coming of the surface operator 



^r4,o,5D(^)= Yl ^"^ 



^^^^^^''^z^cct.,5D{a, Y; t, q) JJ 2fund.,5D(a, Y, mj^t, q) 



1 



(3.23) 



where we introduce the mass parameters 

7713 = rn — TTT-i + e, m4 = m + rfii + e. (3.24) 

This formula is the five-dimensional lift of the result of [15] . While the four-dimensional version of 
(|3.23p was derived from the conformal block via the AGT relation in |15] . we employ the topological 
vertex for engineering the ramified instanton partition function. 

Now let us move on to make the meaning of the specialization p.lSp and (j3.19p clear. The 
point is that the toric brane realization of surface operators leads to these special values through 
the open/closed duality. Let us consider the system which is drawn on the left side of Figj9j As 



/\ 



sK' 



y\ 'J— 



Y > 



-< ^ 



Figure 9: The geometric transition relates the surface-operator/toric-brane insertion and the bub- 
bling geometry which engineers the SU{2) x SU{2) linear quiver gauge theory Ti^o- 

we have discussed in the previous section, the Lagrangian brane denoted by the broken line in 
Figl9] engineers the surface operator. Here we introduce the framing in order to compactify the 
Lagrangian cycle. Such a compactification does not essentially change the amplitude since the 
A-model topological string theory is independent of the complex structure moduli. Then we apply 
the geometric transition [4^ which changes branes into the background geometry. The multiplicity 
of branes then becomes the Kahler parameter of the emerged cycle. The geometric transition 
for our system is shown in FiglH Since we have inserted only one brane on the upper leg as 
in Figini a new CP whose size is ti = gs emerges into the toric geometry after the transition. 
There is also the Lagrangian cycle 5^ without a brane. This Lagrangian three-cycle gives the zero- 
size CP through the transition. In this way, the geometric transition relates the two reaization 
of the surface operator: the open topological string approach and the closed topological string for 
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3/2,1/2 



1/2,1/2 



q-''H 



1/2^1/2 



g-1/2^1/2 



Figure 10: The toric geometry which engineers the SU{Nc) Nf = 2Nc gauge theory in the presence 
of the elementary surface operator. 



degenerate parameters. The refined version of the brane-induced parameters ti^2 = 5s; is precisely 
equal to (I3.18P and (I3.19P which were derived from the CFT side. 

The phenomenon which changes branes into two-cycles is known as the bubbling Calabi-Yau 
[50^151]. The field theory limit of such a duality then proves the implication of the AGT relation: the 
ramified instanton partition function for an elementary surface operator emerges from the Nekrasov 
partition function by limiting the parameters. This feature provides an efficient tool to compute 
ramified instanton partition functions. 



3.3 Generalization to SU{Nc) gauge theory with 2Nc flavors 

We can easily generalize this result to the 74,o[^Afc-i] theory. The toric geometry for the SU{Nc) 
gauge theory can be obtained by gluing the geometries for SU{2) theory in vertical direction. See 
|49] for details. We choose the degenerate parameters as in FigllOl The refined vertex on the 
strip geometry then leads to the following factors for each 0{—l) ® 0{—l) curve inside the strip 
geometry 

JJ (1 - f-ig-^+2) J] II (1-f-ig-^+i). (3.25) 

{i,j)eWi a=2 {i,j)eWa 

The Young diagrams take the following form give the non-zero contribution for the partition func- 



tion: 



1^1 = [1"] in = 0,1,2,- ■■), W2,3,... 



Nc 



<P. 



(3.26) 



Therefore the toric geometry FigJTOl engineers the ramified instanton partition function for the 
SU{Nc) gauge theory with 2Nc flavors. 
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Figure 11: The toric geometry which engineers the M = 2* gauge theory 7i,i. The horizontal 
direction is periodic by the identification of edges. 

3.4 Ti,! : AT = 2* SU{2) gauge theory 

Let us next consider the M = 2* SU{2) gauge theory: the SU{2) gauge theory with an adjoint 
matter. The toric geometry for the theory is ihustrated in Figllll The refined vertex on the 
geometry leads to the Nekrasov partition function for the M = 2* SU{2) gauge theory: 

It is easy to rewrite it into the standard form of the Nekrasov partition function by using the 
formulae in Appendix B. Here the identification between the Kahler parameters and the gauge 
theory parameters is 

Q/,Q™.=e-^^^"^/V/2, (3.28) 

Q™, = Qm.2 = e^'^q-^lh^'\ (3.29) 

See [20] for the detailed study of such a partition function. 

72,1 '• ^1 quiver gauge theory 

As we have mentioned at the beginning of this section, the Ai quiver gauge theory leads to the 
surface operator of the M = 2* theory via the AGT relation. This quiver gauge theory is also 
realized by the string theory on the toric Calabi-Yau Figll21 The refined vertex on the geometry 
implies the following partition function: 

7 _ v^ r^\Yl\+\Y2\A\Wl\+\W2\ ^ yi.F2.w^2^,Wi^(^' ^) ^ Wi.ty2,y2^,yi^(g' ^) /o qn\ 

1^1,2, W12 
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By using the formulae in Appendix B, we can show easily that this is equal to the five-dimensional 
Nekrasov partition function for Ai theory 



"^ii,5D 



Yi\ + \Y2\ Q/\Wi\ + \W2\ , 

X 2;bifund,5D(«, y-, b, W; m)zu{und,5D{b, W\ a, Y; m'). 



Yi,2,Wl,2 



(3.31) 



Here M is a monomial which is irrelevant in the four-dimensional limit. Now let us verify our 





Figure 12: The toric geometry which engineers the Ai gauge theory 72, i . 

expectation that the ramified instanton partition function for the M = 2* theory is equal to the 
specialization of (j3.3ip . Before we turn to the derivation from the string theory, we study the 
Liouville CFT side to find the corrsponding degeneration of parameters. 

Conformal block for 72, i theory and its degeneration 

Recall the AGT relation for the 72, i theory: the conformal block |52t [53l [5^ [55] for torus with two 
punctures Figll3lis the counterpart of the Nekrasov partition function for the Ai gauge theory. We 
can compute the conformal block perturbatively as 



Br2,ii^i,Q2) = 

^ , (Ai-Aa-Afo)(A2 + A„- Afe) {Ai-Ab-Aa){A2 + Ab-Aa) , 

-L + Ql ;tt r (?2 ^TT r 



2A„ 



2A, 



(3.32) 



The masses of the Ai theory relate to the external Liouville momentum of the conformal block 
eie2Aj = mi{e — rrii). The U{1) part of the Nekrasov partition function, which is a crucial factor 
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A, 



Figure 13: The conformal block for torus with 2 points, which corresponds to SU{2) Ai-quiver 
gauge theory. 

for the gauge/Liouville correspondence [1], is given by 

oo 

1=0 

X fl _ Q^+lQ^')2m2(e-mi)/eie2Q _ „^ „^+l\2mi (e-m2)/eie2 

2m2(e-mi) 2mi(e - mg) , . 

= l-qi 92 ^ ■ (3.33) 

eie2 eie2 

From the perspective of the AGT relation, this factor comes of the free-field computation of the 
corresponding conformal block [52j. Based on explicit computation, we can confirm that the product 
of these factors Z^^^' -B is equal to the Nekrasov partition function for the SU{2) Ai gauge theory 

in- 

Let us consider the degeneration of the external vertex operator V^jl^a) — >• ^2,i{(l2)- The fusion 
rule then implies the following specialization of parameters: 

«' .2 A _^' (. . ^2\2 ^ _ eie2 36i 



Aa = j-a^ A, = --^a + ^j , A2 = -^ - ^ (3.34) 

Notice that this situation implies the following specialization of parameters of the gauge theory 

1712 = - — , b = a + —. (3.35) 

Then the degenerated conformal block becomes 

Br.Agu,2) = l-<li ^^^^^^^ + 92 ,^^2a-e,) + •••• ^^'^^^ 

ryU{\)i s 1 (e-mi) mi(2e + e2) . . 

Zr:'{qi,q2) = 1-91 92 + ••• • (3.37 

The product of them is the partition function for the surface operator since the degenerate field 
corresponds to the operator via the AGT relation. These expressions thus lead to the ramified 
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instanton partition function for the M = 2* theory: 



U{1) 



vI/r„(gi,Q2) = ^^^;/er.,, 



^ , {e-e2/2-mi){2a + 6 + 62/2 -mi) 
J^ + '71 — — : — ^ H 



(3.38) 



ei(2a + e) 

We can also recover this result by computing the corresponding refined vertex amplitude ()3.3ip for 
the following degenerate parameters (j3.35|) . 



Degenerate Kaher moduli and bubbling 



Let us interpret the degenerate parameters (j3.35p . We saw in the case of the Nf = 4 theory that 
the toric brane construction leads to such a degenerate Calabi-Yau geometry after the geometric 
transition. This idea works also for the M = 2* theory. The transition of the elliptic geometry 
for the M = 2* theory is shown in Figll4[ From the viewpoint of the geometric engineering of the 
72,i[^i] theory, the degenerated parameters (j3.35p correspond to the special choice of the Kahler 
parameters: 



Qm,i — ^y 7' Qm,2 — \ J- 



(3.39) 



This specialization reflects the A-brane which was inserted in the upper leg before the transition 
as in Fig mi 





Figure 14: The toric geometry which engineers the SU{2) x SU{2) Ai quiver gauge theory 75,o- 
In this way the five-dimensional lift of the ramified instanton partition function is 

'^M=2*{q,z) = ^ Qi'^a '2;vect,5D(a,i^)2vcct,5D(^ = « + £2/2,1^) 
Y,W 

X 2bifund,5D(a, Y]b = a + 62/2, W; m)zbifund,5D(ft = o + £2/2, W; a, Y; 7712 = -£2/2). 

(3.40) 

Here qi = q^z~^ and g2 = —(f'z. It is easy to take the four-dimensional limit i? — ^ 0. We can see 
that the four-dimensional result equal to (13.380 based on the explicit computation. 
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3.5 Decoupling limit and pure SU{2) Yang-Mills theory 

So far we studied superconformal gauge theories. Finally let us consider asymptotically free theory 
|561I57|. For our result (|3.40p . we can take the decoupling limit of the massive hypermultiplet. We 
propose the following decoupling limit 

m^oo, m^qi-f K^\~^, rr?q2^-\. (3-41) 

Here A = A^z is the two-dimensional instanton factor [581 [33]. Then we find the following expression 
for the ramified instanton partition function of the N = 2 pure SU{2) Yang-Mills theory: 

^pureSYM(^i,^2) = Yl i^^ >^~^f\- >^)^^^ z,,,t{a,Y)z,cct{b = a + e2/2,W) 
Y,W 

X ^bifund(& = a + £2/2, W; a,Y;m = -£2/2). (3.42) 

One can compute the four-dimensional case by replacing the instanton measures as follows: 
z^ect(a,f)^ Yl H {aa - ab + ei{-Yi^ + i) + e2iYai - j + l))y^ 

a,b=l,2 {i,j)eYa 

X n («a-«fc + ei(^ai-^ + l) + e2(-n*+j)))"\ (3.43) 

zuinnd{a,Y;b,W;m)^ H JJ {ua - h - m + ei{-Wifj + i) + e2{Yai - j + 1))) 

a,b=l,2 {i,j)&Ya 

X n iaa-bb-m + eiiY;[j-i + l)) + e2i-WM+j)), (3.44) 

where ai = —02 = a and fei = —62 = b. 

Recently the author and Maruyoshi [33] proposed the irregular conformal block for the ramified 

insanton partition function: 

00 

'^purcSYM(gi,52) = 5]5]A2|^l+2"z""l^l/3};QX'([l"''];n 
n=0 Y 

= 1 + A . } .,. +A^A"^ ^ 



2ei(a-f) 2ei(a + f + ^) 

+ A'^T7 ^Y7 -,+■■■' (3-45) 

8ef (a- ^) (a-ei) 

where /3 is an expansion coefficient of the degenerate field on the Verma module: 

'^2,i{z)\A,Y) = J^zl^'l-ivi+^^jy |A',y'). (3.46) 

Y' 

Our formula (13.42P in the four-dimensional limit i? — t- will provide the combinatorical expression 
for the representation-theoretical expression (j3.45p . We can see the agreement of these results based 
on the explicit computation. This relation between (|3.42p and (13.45P provides the extension of the 
asymptotically free AGT relation [561 EZ| for surface operators. 
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4 Conclusion and Discussion 

In this work we study surface operators by using topological string theory and dualities. Our study 
suggests that topological string theory provides the powerful way to reveal the mechanism behind 
the mysterious AGT relation. 

We study the geometric engineering of the Nekrasov partition function in the presence of the 
surface operator, which we call the ramified instanton partition function. We find that a toric 
A-brane corresponds to a surface operator of gauge theory through string dualities. We show that 
we can derive Gaiotto curves from the wave functions for the open topological string amplitudes 
based on the idea. This approach will be an alternative to the matrix model computation of the 
B-model in |15| . 

We also show that the insertion of surface operators leads to the bubbling of Calabi-Yau ge- 
ometry. By using this idea, we compute the ramified instanton partition functions for the A'^^ = 4, 
M = 2* and pure Yang-Mills theory. Our approach provides an efficient way to compute the 
ramified instanton partition functions for elementary surface operators and their five-dimensional 
lifts. 

There are many open problems related to our study. For example, it would be very interesting 
to develop topological string realization for more generic type of surface operators such as the full 
type in [59]. Since these operators appear when we study SU{N > 3) gauge theories [Ml El], we 
would get the idea from topological strings on toric geometries with long strip. It would be also 
important to study relation to the Hitchin systems [Ml [M], which are expected to play a key role 
in the AGT relation. We want to come back to these problems in the future. 
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Appendix 

A Young diagrams and Schur functions 

A Young diagram is a sequence of decreasing non-negative integers such that 

Y = {Yi£Z>0\Yi>Y2>---}. (A.l) 
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The transpose of a Young diagram Y is defined as follows 

Y^ = { y/ G z>o| y/ = # { i| y, > j}} . (A.2) 

We defines the following quantities 

d(Y) d{Y) 

|y| = 5;y, ||yf = ^y.2, ^y = J^ (j-i). (A.3) 

Note that we have ny = \\Y\\ — \\Y'^\\ . 

The Schur functions are symmetric functions which are labeled by the Young diagrams. It is 
the trace of X = diag(xi, X2, • • • ) in a representation Y 

Sy{x) = TtyX. (A.4) 

See [To] for details about the symmetric functions. We can also define the skew Schur functions 
^Yi/Y2 = Sy Cy y'S'y where c is the coefficient Y2 ^ Y = "^ Cy^ yYi. These functions then satisfy 
the following properties 

SY{Qx) = Q\^\SYix), (A.5) 

5y/v.(Qx)=Ql^|-|^l5y/y(x), (A.6) 

SYiq") = q'^Syriq") = {-if^YyTiq-")- (A.7) 

Here g^ denotes the variable Xj = q"'^^^''^. The following identities, which are known as the Cauchy 
formulae, are most useful for the computation of the topological vertex 

Y, SY{x)SY{y) = n (1 - ^*2/i)"'' (^-8) 

Y i,j 

Y,SYT{x)sY{y) = n (1 + ^'2^j)' (^-9) 

Y i,j 

Yl SY/vix)SY/wiy) = Yli^ - XiVj)'^ Yl Sv/Yiy)Sw/Yix), (A.IO) 

Y id Y 

Y^YT/vi.x)SY/w{y) = W{'^ + Xiy])Y^y/y'^^y'>^wT /yt{x). (a.ii) 

Y ij Y 

We make good use of these formulae in order to compute a vertex on a strip. See [iHl 08] for details. 
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B Five-dimensional Nekrasov Partition Function 

B.l Instanton partition function 

In this section, we explain how the Nekrasov partition functions are derived. The point is that 
an instanton partition function becomes more simpler by taking the five dimensional gauge theory 
with twisted boundary condition (if, X4) ~ (expRQ ■ x,X4^ + R), where R is the radius of the fifth 
dimensional S^ and fi is a generator of the rotational symmetry of the space-time M^ x M^ 

n=\ . (B.l) 

y €2! 

Vector multiplet 

The fc-instanton part of the five dimensional pure super Yang-Mills theory reduces to the super- 
symmetric quantum mechanics on the instantin moduli space Mk by taking small coupling limit. 
The instanton partition function is then defined as the Witten index of the quantum mechanics 
which is the index of the Dirac operator. The Atiyah-Singer index theorem leads to the following 
expression: 

Zpur:e,k;5D = IndP 

= [ A{TMk). (B.2) 

JMk 

Here A is the A-roof Dirac genus 

^^TM) = n ^^^,, _ l_^,, (B.3) 

where the Chern roots are ch{TM) = X^j e^\ This k instanton partiton function is just the 
arithmetic or Todd genus of the moduli space except an irrelevant coefficient 

^pure,fc,5D = Y^i-irdlmcH^'^'iMk). (B.4) 

n 

In order to take the twisted boundary condition into account, we have to evaluate the following 
partition function by using the equivariant localization theorem 

°° r Rt 

fc=0 •^•^fe i ^ 

It is known that the U{1) "^^ x {/(l)^, which is the maximal torus of the isometry of the ADHM 
instanton moduli space, enable us to apply the equivariant localization method to compute this 
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partition function. The Duisterniaat-Heckman formula leads to 



fc=0 p:fixed point i 



where Wi^p are weights of the torus action at the fixed point p. Fixed points of the torus action 
C/(l)^'=~^ X f/(l)^ are labelled by the Young diagrams Y = {Yi, • • • , Y^J which satisfy Yi + ■ ■ ■ + 
Y]\f^ = k. For the fixed point Y, the weights were computed by Nakajima [67] as 

iVc 

^g-H^-,,,?)^ ^ g-R(a„-a,)J^ ^ <^-n^+^t-n,+i-l+ ^ g^5-^+lt^^--i), (B.7) 

J a,b=l ii,j)eYa ii,j)eYi 

where q = e~^''^ and t = e^''^. There are thus 2A'"c|y^| = 2kNc fixed points which contribute 
to the instanton partition function via the localization theorem. Notice that this number 2kNc is 
precisely the dimensions of the ADHM moduli space of fc-instantons. By substituting these weights, 
we arrive at the partition function of the M = 2 pure super Yang-Mills theory [Sj [Ml [65tl66j: 

_ _ _ _ -.is — - n — 

where Qah = e"^*-""""''. The contribution of a vector multiplet is 
Hyper multiplet 






1 1 

z.ect.Ma,Y;q,t)= TT TT -^- JT -^— . (B.9) 

-'--'- -'--'- 1 /O — K +2 j._V -1-1 — 1 J-J- 1 /-> 5^ — «+l J.VL — 1 



In order to introduce matter hyper multiplets, we consider the matter bundle E over the instanton 
moduli space Aik whose fiber is the Dirac zero modes on the instanton background corresponding 
to the base point of A4k- Then the instanton partition function is the index for the twisted complex 

Zk-5D =IndP£; 

= / A{TMk)ch{E). (B.IO) 

JMk 

The matter contribution therefore leads to such insertion of the equivariant Euler characteristic 

into the partition function. 

Since the adjoint matter bundle is just the tangent bundle to TWfc, the instanton partition 

function for J\f = 2* theory is 

^adj,5Z, = E^V 11(1 - e-«-e-«-)3-^p^ = E'^'X.(A^.), (B.ll) 

fc=0 •^•^fe i k=0 
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where y = e~^"^. This is precisely the Xy genus of the nioduh space. When m = the A:-instanton 
partition function Xy reduces to the Euler characteristic ^(— l)"'™dinic -ff"'''"(A^fc)- Note that 
this hniit gives the partition function for Af = 4: Yang-Mills theory: 

oo 

Zu=A,5D = Y.q''xiMk), (B.12) 

fc=0 

Let us compute the instanton partition function of the M = 2* theory with nonzero mass. The 
Duistermaat-Heckman formula leads to 

CXD 

fc=0 pgfixed points i 

By using the Chern character ()B.7p . we arrive at the instanton partition function of the J\f = 2* 
theory. The contribution of the adjoint hypermultiplet is 

a,b=l (i,j)eYa 

^ n (l-e~^^''-~'"'""'^q^-^-'+H^>'^-A , (B.14) 

We can also discuss the contributions of matters in other representations using the localization 
method. The results for the fundamental and bifundamental representations are 

Ziun±Ma,m,Y;q,t) = ll J] (l - e'^^"'^''"'")*?*-! t-^+i) , 

a=l {i,j)&a 

Nc 
^bifund.,5D(5, Y- b, W;q,t)=Yl H (l - e-^('^-^-'")g-^5+^ ^-^.+.-1) 

^ n (l-e--^("''-'"'-™)/5-^+it^w-iV (B.15) 

When one consider a gauge theory with matters in these representations, one can construct the 
partition function by using these factors. 

B.2 Useful formulae 

Let us introduce the following combinatorical factor 

Ny^xMi^t) ■■= n (l-Q^z'^^^^^^t'^" (*)+') n (l-Qg-'^^<'(*)-^t-'^''(*)) . (B.16) 

seYa teYt 
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Note that this is a basic building block of the instanton partition functions. The following identities 
are very important and ubiquitous in the study of the Nekrasov partition functions: 

S&Yt t£Ya 

".-Li l-Qqi-Hi 

Under the transposition of the Young diagrams, this satisfies 

iVy.,n(Q"^g,i) = A^yT,yT(Q;t,g)^^ (^Q^^J , (B.18) 

where we introduce the modified framing factor of [H] fviq^t) = { — ly^'q''^'' '^t~ll^ II '^. We 
then get the following property of the vector multiplet factor: 

n NY^,niQab;t-\q-') = n NyT^yT{Qab;q,t) =(-] n NY.,n{Q-,';t,q) (B.19) 

a,b=l a,b=l ^^^ a,b=l 

Consult the extensive study of Awata and Kanno [681 [69] for proof of these identities and other 
combinatorical properties of the Nekrasov partition functions. 

By using these formulae, we can derive the following expression for the instanton measures |48j : 



z^,ct.MS,Y-q,t) = (-l)^^l^l - ly-rv oa+i^||v-i|2 n Q'^b 






\ya\+\yb\ 



Nc „., „2 



X 

a=l 

. n n '-'^if'''" n '-"^if'Z' (b-^o) 

It is this identity of which we have to make good use in rewriting the refined topological string 
amplitudes into the Nekrasov partition functions. 

The instanton measure of hypermultiplets also have infinite-product expressions: 

a,b=li,j=l ^ ^ab^m q ^ 

-ft ^ 1 - e-«(-«''-™)g^5-rJ-+i 
^fund.,5D(a,m,y;g,t) = || [[ ^ _ ^_^(_,^_^) .^_^.^, , 

a=l i,j=l 

a':,tlijtl ^ - QabQm q-'+H-^ 
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We utilize these formulae for rewriting topological string partition function into Nekrasov instanton 
partition functions. 

B.3 Refined topological vertex 

Inspired by the connection between the topological vertex and the Nekrasov partition functions |21j . 
some groups seek an extended topological string theory corresponding to the general 0-backgrounds. 
The refined topological vertex which was proposed by Iqbal, Kozcaz and Vafa in [26] was based on 
the melting crystal dual picture of the A-modeo The precise form of the vertex function is 

CYiY2Y3{t,q) 

11^211^+11^311^ ^y |y|+|yii-|y2i 

= (f) ' t^Py^T{t-'>;q,t)Y,{^) " Sy^T,y[rPq-''-^)Sy^,y{t-^-^\-''), (B.22) 

Y 

where PyT{t~^;q,t) is the following specialization of the Macdonald function [701168] 

PyT(t-^;(Z,t) = t^ll^ll'Zy(t,g), ZY{t,q)= n (l-t''^"-'+'/'-^r^ (B.23) 

The arguments x = t~Pq~^ denote Xi = f~^''^q~^^ for i = 1, 2, • • • . 
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